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SECRET § March 1973 


) 


MEMORANDUM FOR THE RECORD 


SUBJECT: Task Deception for Product Management Part 
of the DOUL/iU otair 


This memorandum further describes ideas for the organization of the 
Product Management Staff I presented briefly under Alternative I in 
an carlier paper. Overall, this part of the DCI/IC staff should be 
the principal force in increasing and maintaining product quality 
within the intelligence community. The efforts of this part of the 
staff go beyond a narrow view of quality control. The results of 
quality control efforts have been to sterilize the product rather than 
make it more useful in many cases, Quality control has been fre- 
quently misinterpreted as being standardization, It has resulted 

in the creation of a standard set of qualifying words, a standard 
writing style, rules and procedures about coordination and agreed 
positions, anda limit on experimenting with new techniques of 
analysis and presentation, Product quality must be keyed to the 
consumer needs and diversity of conswrers requires a system which 
allows diversity of output. 


STAFF SECTION: NEW METHODS AND EXPERIMENTAL PRODUCTS 


This staff section is responsible for R&D in new methods of analysis 
and presentation and for developing experimental products to test 
these techniques. R&D has to be a two-fold approach, First, there 
must be a continuous effort to develop methods to analyze problems 
and to test their applicability to intelligence production, Second, 
research needs to investigate how the results of the analysis can be 
best communicated to the consumer. 


lL. R&D in Analysis 


Research in analytical techniques is virtually non-existent in the 
community, Work in organizational analysis and Bayesian 
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techniques, for example, has been instituted from the outside 

and has encountered strong resistance. The use of management 
science tools has been recognized, but probably because these 

can be seen as mechanical tools, YThought-provoking techniques 
seem to threaten the analyst's individualistic, intuitive approach 

to intelligence, to unacceptably expose managers to the risk of 
making ‘wrong’ estimates (i.¢e., those less defensible to superiors), 
and to interfere with the production-line approach, 


DCI level initiative is needed to give visibility and support to R&D. 
There are disadvantagea in that lower-level managers may be 
antagonistic to upper-level initiatives and that a lower-level 
approach (DI and DDI) would bring the R&D task closer to those 
daily concerned with products, However, I would prefer to see 

the involvement of subordinate elements grow gradually from 

the DCI's activities, Eventually, the insertion of new organizational 
elements, not under the direct control of production managers, 

may be required to provide appropriate incentives to the managers 
of the R&D effort and segregate their resources from those devoted 
to production, How te organize the community to do R&D in the 

long run is an important question, Contracting on a long-term 
basis, in order to build up high quality analysis groups, away from 
the day-to-day production treadmill where they can try new approaches 
may be useful, 


A four or five man staff could: 


~~ Conduct surveys of new methods and assess them in 
a preliminary way. 


-- Develop programs to have appropriate R&D performed. 
~- Monitor the R&D programs 


-- Supervise testing and trial applications in production 
Pp PP P 
organizationg, 


2, R&D in Presentation 


The community's output is frequently insensitive to consumer 
needs because products are not designed with an eye to the time 
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consumers have to study them, the manner in which they like to 
receive information (c.9., Kissinger dislikes briefings) and the 
issues which need to be hishlichted for their attention. I 
recognize that the agencies serve a number of consumers in 
addition to thoge at the national level, however, these problems 
probably exist elsewhere, Getting consumers to give guidance 
is also a problem, but the community is not making enough. of 
an effort on its own to use presently available contacts and 
Buucctes tu ldvulily buciy ucedy (s66 my cornments on consumer 
research). 


Iam somewhat unsure where you should place the staff responsi- 
bility for improving presentations. It is linked closely both to 
research in new methods and to consumer research, I prefer 
locating it with the former because new techniques (e.¢g., Bayesian 
analysis) generate new manners of presenting intelligence. Also, 
experimental products which test new presentations are closely 
yelated to those testing new methodologies, Some experimental 
products which you may investigate are primarily new executive 
summaries, new ways to communicate uncertainty and even more 
unusual approaches such as cassette recorded briefs for dis- 
sermination to busy consumers, 


STAFF SECTION: CONSUMER RESEARCH 


Present efforts to identify consumers needs have been unsystematic. 

My staff has tried to provide feedback based on our experience with 
consumers in the White House and NSC, I have interviewed a number 

of NSC staff members. Several organizations -- ONE, DI -- have 

made an effort to tailor their products to some general consumer needs, 
Most, however, have continued to roll out products much like an assembly 
line, and some have been particularly unreceptive to feedback, to say 
nothing of using it. 


The consumer research group would have three basic jobs: 
-- Search for techniques to use in identifying consumer needs. 
-~ Disseminate end encourage agencies to use these techniques, 
~- Use these techniques to ascertain the needs of national- 


level consumers whom the DCI supporta directly. 
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Getting feedback from consumers is obviously a difficult task, Users 
generally are too busy to respond thoughtfully and they are more likely 
to respond favorably because they do not have time or desire to be 
critical. However, intelligence has not been very diligent in pursuing 
consumers and not very creative in reaching them, A consumer 
yesearch ataff could tan a variety of sources in business (especially 
the publications industry) and academic circles (business schools) for 
market research approaches, Outside assistance could probably be 
sought without incurring problems in security clearances, Also, I 
can suggest several ideas: 


1. Extremely brief ‘'yes-no!' or ''scale' questionnaires could 
be used to reach busy people. Essential information could be 
derived suchas: has he read the document; did it add to his 
knowledge: were the points clear; and, did he find the paper 
valuable enought to keep’ 


2, Questionnaires of varying lengths could be used for different 
consumers, for example, one type of questionnaire could be veed 
for analysts, one for staffs supporting major consurrers, and 

one for unfamiliar third parties (e.¢g., consultants), 

3, Interviews of varying types can be conducted. Several readers 
could be interviewed in depth; many could be simply called on the 
phone and asked a few unclassified questions. Sampling could be 
used to reduce the volun e of work, 


4, Face-to-face conversation with cach consumer is not critical, 
A great deal can be learned about a man's needs by talking to 
those around him and by speculating about the type of work he 
does and what he needs to doit, In this sense a consumer's 
speeches, publications and otner reflections of his demand for 
information could be studied. 


5, A program of experimental preducts along with close monitoring 
of their reception is another major way of testing consumer needs, 


PFlexibility is critical, Questionnaires should add or delete items as 
issues gain or lose importance. Folling should initially sample the 
general views of people, then find key problemis and pursue these in 
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greater detail, Short and long questionnaires and different styles of 
interviewing should be intermixed in order to keep the effort alive, 
Different approaches should be used for the different categories of 
publications -- i,e,, current products {e.g., CIS) versus estunates 
(e.g., NIE's} versus special products (e.g., weapons studies). The 
research effort need not be a statistically intense, scientifically 
oriented task. Since the present efforts of the cormmunity are so 
underdeveloped, [think it is far more important that an aggressive, 
common-sense approach toward eliciting consumer reaction be taken, 
A weil maneged effort will easily surface major issues and problems 
in consumer demand, 


START SECTION: PRODUCT EVALUATION 
The community's present output needs constant review and evaluation, 
Judging by the products I have seen and the interviews I have had with 
consumers, the intellectual quality and appropriateness of many 
products are questionable, Not only are they insensitive to consumer 
needs, but they lack real substance. Many recite facts gained from 
collection systems with no real analysis; hedging muffles the message; 
isolated facts are tied together with simple inferences that could be 
done by practically anyone; and cliches and dubious assumptions 2bound, 
The agencies have been resistant to criticisms suchas these, arguing 
that real analysis has been done but is not presented in order to pro- 
tect the consumer's time, or they have simply been irritated at being 
evaluated, 


The principal responsibilities of this section would be (a) to conduct 
evaluations of products in general and of those destined for national 
consumers in particular, and (b) to encourage each agency to develop 
and improve its own system for product evaluation, The principal 
tasks which the group would face are: 


.- Investigating quality control experiences in industry, 
academic institutions, and other government organizations 
for ideas, 


-- Defining various measures of product quality and their 
applicability to community outputs. 


.. Establishing means by which products can be judged 


and the results sent to appropriate individuals. 
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-- Developing procedures to insure that, based upon 
feedback, changes in preducts take place. 


-- Gonducting evaluations of specific community outputs 
and of products (e.g,, NIE's) going to national level consumers, 


Considering the qualitative problems and the community's capacity to 
resist, rendering critical but constructive evaluations will be an 
extremely difficult task, Performing perceptive evaluations is 
intellectually demanding in its own right. Submitting these evaluations 
to individuals unaccuatomed to such feedback, and convincing them of 
the need to change, is doubly difficult, Defining quality will be a 
continuing challenge, The view of 'what is quality’ varies among 
producers and users; an. understanding of these different views is 
important in determining what influences need to be employed to im- | 
prove output, The definition of quality also covers a number of 
criteria -- ce. %., the length of publications, quality of English grammar, 
number of products, response tirne to questions, analysts" intuitive 
satisfaction, and the presence or absence of feedback, Each criteria 
has some merit in the perspective of different participants, and all 
have some validity in an absolute sense, 


A main cffort of this section should be to work out the qualitative 
methods of evaluation, For example, the depth of analysis in products 
will need to be assessed, the mix of data and condusions, the systematic 
presentation of speculative explanations, etc. Doing this is going to be 
far move threatening to analysts' pride and sense of competence than 
more mechanical controls, Hence, there will he a need for very 
competent and articulate evaluators. Consultants and outside experts 
may be used in panels or other ways to assist in judging output. 
Sampling could be used to keep the section from being swamped 

with products. 


STAFF SECTION: PRC AND SU MONITORING 


More than half the effort of the intelligence agencies is devoted to these 
two nations. For this reason alone I think that two sections at the DCI 
level should be created to monitor the entire intelligence effort devoted 
tothem, These sections should not control the activities of various 
agencies, which would interfere in the organizational prerogatives of 

. ynanagers, Their tasks would include: 
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+ Providing the DCI with recommended guidance to be 
implemented through the USIB and other interagency forums. 


_. Monitoring the estimates and other products which are 
forwarded to national consumers, 


-- Expediting and representing the DCI in dealing with 
new or major issues {(e.g., PRC migsile location). 


.~- Improving the coordination among different agencies 
and the use of collection, processing and production 
resources, 


In essence, these groups attempt to coordinate the strategies of all the 
diverse elements of the community which follow the 5U or the PRC, 
Their leverage in the process rests upon their informal influence, 
their ability to move among all agencies, and their capability to 
influence the DCI's decisions, In addition to studying the efficiency of 
the community's operations and evaluating their products, the section 
would have a role in the budgeting process, assisting the DCI controller 
in evaluating the impact of changes in funding. 

STAFF SECTION: SPECIAL PROJECTS 

This might be a small group of individuals dealing with particular 
projects which arise from time to time, For example: 


.- Monitoring the economic intelligence effort or other 
problem areas in the same rranner as the PRC and SU 
are watched. 


~+ Managing product reviews and coordinating the efforts in 
the NSCIC forum, 


-~ Guiding the preparation of papers addressing major issues 
or breaking new ground in intelligence analysis (e.g, new 


in-depth long range studies of international affairs). 


Ina sense, this is a catch all group, It provides a pool of manpower to 
which important projects can ba assigned which cross several otner 
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sections or have a unique sensitivity, It can manage relationships with 
the NSCIC, the USIB and other bedies. It can help other staff sections 
with particularly difficult enterprises, 


CONCLUSION 


Your staff hag a dual role, It supports the DOI directly in his efforts 
to satisfy the needs of national consumers, Also, it establishes and 
encourages agenctes to implement systems which will enable them te 
conduct their own quality control and product improvement programs, 
In doing so, it monitora the general quality of intelligence products. 


I proposed discrete sections for each major function, Many of these 
tasks are new to the community, and, therefore, ill-defined. Some, 
such as product evaluation, are threatening and intelectually difficult, 
All are going te encounter various forms of resistance, Consequently, 
I feel separate sections will give each function the visibility it requires 
and will communicate toa staff members a clearer idea of their jobs. 


The hazard of such separation is overlap and a major need for coordi- 
nation, I recommended in my previous memo the establishment of 
standing groups or the asgignment to one individual of the responsi- 
bility fer coordinating all the work of the DCI staff in important areas, 
Similar coordination will be needed within your staff, 


Personnel selection is critical, The diversification of the staff to include 


more than CIA personnel has already been diacussed, I think it is im- 
portant to realize that different sorts of people are needad for different 
tasks, Chooaing all old intelligence hands, or experts on particular 
areas, can pose real problems in introducing new techniques or 
recognizing problems, Younger, though inexperienced, people have 

a shortcoming in knowledge, but a freshness of view. It seems to me 
that a mixture of the two is much needed in the DCI's staff, 
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